Skip to content

Show the Policy spellings that type-check - #334

Merged
derek73 merged 1 commit into
masterfrom
docs/type-clean-policy-idioms
Aug 6, 2026
Merged

derek73 merged 1 commit into
masterfrom
docs/type-clean-policy-idioms

Conversation

@derek73

@derek73 derek73 commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Summary

nameparser ships py.typed, and every Policy configuration example in the docs needed a # type: ignore[arg-type]. This changes the docs to teach the spellings that check clean.

Five spellings failed, not the two originally noticed:

documented why it failed
Policy(script_orders={}) dict → tuple
Policy(segment_scripts=()) tuple → frozenset
Policy(nickname_delimiters={("[", "]")}) set → frozenset
Policy(maiden_delimiters={("(", ")")}) set → frozenset
Policy(script_orders=dict(DEFAULT_SCRIPT_ORDERS) | {...}) dict → tuple

The bare set literal for a delimiter field is the one most likely to bite in practice — it is the spelling anyone reaches for first, and it appeared in concepts.rst as the introductory example.

What is not changed, and why

The annotations. A dataclass field annotation types both the __init__ parameter and the attribute, and Policy normalizes in __post_init__ (its validators take object), so those two types genuinely differ. Python cannot express that with one annotation.

The annotation stays the storage type, because reading the attribute is the commoner operation and a union there would cascade to every reader — the same reasoning that already kept empty_attribute_default narrow. This is a real expressiveness gap, not sloppiness, and it applies to every normalizing field.

Union spellings were already fine and are untouched: frozenset | set is frozenset, so DEFAULT_NICKNAME_DELIMITERS | {("{", "}")} always checked clean.

The one spelling that needed a contract first

The script_orders override becomes:

Policy(script_orders=(*DEFAULT_SCRIPT_ORDERS, (Script.HAN, GIVEN_FIRST)))

which depends on a later entry replacing an earlier one. That behavior was emergent — _validated_script_orders builds a dict before sorting, and the comment there justifies it for hashability, not for override semantics. Prose telling readers to append is prose that depends on it, so tests/v2/test_policy.py now pins the override, the survival of the other entries, one-entry-per-script, and equality with the mapping spelling it replaces.

Verification

  • Every spelling the docs now show, extracted verbatim into one file: mypy clean.
  • uv run sphinx-build -b doctest docs — exit 0.
  • uv run pytest -q — 3060 passed (was 3059; one new test).
  • uv run mypy, uv run ruff check — clean.

Note for review

The .. note:: in customize.rst previously told readers the readable spellings need a # type: ignore. It now explains the storage-vs-constructor split instead, and says the wider spellings still parse identically for anyone not running a type checker.

I did not add a release-log entry: no behavior changed. Arguably the newly-contracted last-wins rule is a promise the library now makes and could warrant a line — happy to add one if you read it that way.

🤖 Generated with Claude Code

The package ships py.typed, and every Policy configuration example in
the guide needed a # type: ignore[arg-type]. Five spellings failed, not
the two first noticed: script_orders={}, segment_scripts=(),
nickname_delimiters={...}, maiden_delimiters={...} and the
dict(DEFAULT_SCRIPT_ORDERS) | {...} override. A bare SET literal for a
delimiter field is the one most likely to bite, being the spelling
anyone reaches for first.

The cause is not sloppiness and is not fixed here: a dataclass field
annotation types both the __init__ parameter and the attribute, and
Policy normalizes in __post_init__, so those two types genuinely
differ. The annotation stays the storage type, because reading the
attribute is the commoner operation and a union there would cascade to
every reader -- the same reasoning that already kept
empty_attribute_default narrow.

What changes is which spelling the docs teach. Each one has a
type-clean equivalent that means exactly the same thing, verified
under mypy. Union spellings are untouched: frozenset | set is already
frozenset, so DEFAULT_NICKNAME_DELIMITERS | {...} was always clean.

The script_orders override becomes (*DEFAULT_SCRIPT_ORDERS, (Script.HAN,
GIVEN_FIRST)), which depends on a later entry replacing an earlier one.
That was emergent from _validated_script_orders building a dict before
sorting rather than a stated contract, so prose telling readers to
append needed it pinned: a test now asserts the override, the survival
of the other entries, one entry per script, and equality with the
mapping spelling it replaces.

The note in customize.rst said the opposite of what is true now; it
explains the storage-vs-constructor split instead.
@derek73 derek73 added this to the v2.1 milestone Aug 6, 2026
@derek73 derek73 added the docs Documentation fixes and updates label Aug 6, 2026
@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.48%. Comparing base (57f3b5d) to head (afde560).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #334   +/-   ##
=======================================
  Coverage   98.48%   98.48%           
=======================================
  Files          41       41           
  Lines        2832     2832           
=======================================
  Hits         2789     2789           
  Misses         43       43           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@derek73
derek73 merged commit 65a95e2 into master Aug 6, 2026
11 checks passed
@derek73
derek73 deleted the docs/type-clean-policy-idioms branch August 7, 2026 06:23
sauliusbc pushed a commit to SavantEnvs/python-nameparser that referenced this pull request Aug 22, 2026
Four sites, two kinds of drift.

derek73#334 moved the guide to the Policy spellings that type-check, but only
the hand-written .rst. Four _policy.py docstrings teach the old ones
and autodoc renders them into modules.html, so the API reference and
the guide disagreed on the same page. Worse, derek73#337's new runtime
warning hands the user Policy(segment_scripts=()) -- the library
telling you, in a package that ships py.typed, to write an arg-type
error. A test now pins the offered spelling; the warning tests matched
on 'ja_segmenter' and never checked the actionable half of the
message.

Two stage headers understated their inputs. _extract declared only the
two delimiter policy fields while reading three Lexicon suffix fields
through _suffix_shaped, which is not a detail: that is the mechanism
letting a clause's content overrule the delimiter, and derek73#335 would
extend it. _post_rules omitted Policy.middle_as_family.

Checked mechanically rather than by eye: every stage's declared Reads
against the policy and lexicon attributes it actually touches.
_script_segment, _tokenize, _assign, _group and _classify were
already accurate.

_vocab has no Reads line by design, being a helper whose predicates
take vocabulary explicitly. Its 'no state' claim still holds.
derek73 added a commit that referenced this pull request Oct 27, 2026
…NTS.md files (#909)

* Move the ambiguity-emitter rules from AGENTS.md to nameparser/_pipeline/AGENTS.md

The Ambiguities bullet had grown to 1,488 words in the root file, which
every session loads. It is guidance for work inside the pipeline stages,
so it moves, unchanged in substance and split under subheadings, into a
nested AGENTS.md that loads when a session reads a stage. Root keeps the
principle and pointers to it and to mechanisms.md's contract entry; the
two test comments quoting "Pin the decision, not the vocabulary" now
name the new file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Cut AGENTS.md's design-philosophy paragraph to its doctrine and an index of the script rules

The paragraph had reached 1,458 words, most of it the glued-honorific
peel's comma reach (#312, #319, #613) restated from code. Root keeps the
positional, never-detect-language doctrine, the script-itself exception
and its three-way test, and lists each behavior it licenses with the rule
that states it.

The facts no rule or decisions entry held move to decisions/W2 as a dated
entry, every parse re-run today: the reach and second-condition examples,
where the credential lands by spelling, why the run predicate is not the
token predicate pluralized, what lenient_comma_suffixes=False does, the
four drifts that made the peel ask _comma.decide, and that neither policy
opt-out reaches the peel. The #760 entry, which deferred two of those
facts to this paragraph, now says where they went. The opt-out example is
spelled segment_scripts=frozenset(), the denylist having refused the ()
spelling (#334).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Move the spelling sweep from AGENTS.md to a nameparser/AGENTS.md

The spelling sweep (1,326 words) is needed by any change to a word class,
vocabulary-only changes included, so it moves to a nested AGENTS.md at
nameparser/ rather than under _pipeline/: reading a vocab TOML loads it
too. The text moves unchanged in substance, split under a subheading per
class (numerals, credentials and titles, separators, full stops inside a
word, capitals, adjacent pairs). Root keeps the lead and a pointer; the
Workflow paragraph, which says where a missed shape is recorded, and the
two cases.py notes that cite the sweep now name the new file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix what the design-docs review found in the relocation

- Restore #337's `Policy(segment_scripts=())` in the validation bullet:
  the frozenset() replacement meant for the new opt-out example also hit
  this history line, which records () as the spelling that shipped wrong.
- Retarget the stale pointers: decisions/C14 (honorific-peel paragraph),
  _pieces.py and cases.py:2859 (the spelling sweep's #603 example and
  separator spacing).
- Root index: restore the katakana reason (its script settles nothing,
  not "foreign"), cite W1's interpunct Except for "never segments", name
  decisions/T3 as the nakaguro's history, keep 高橋+一郎.
- decisions/W2: the #760 entry's last pointer, and the facts the cut left
  only in usage.rst or nowhere (#319's failure mechanism, the pre-#319
  readings, why 博士/殿/君 are spaced-only), re-run 2026-10-27.
- nameparser/AGENTS.md: give the moved "That holds" sentence an antecedent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Cite #909 in decisions/W2's 2026-10-27 entry

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Record in AGENTS.md that a whole-file replace rewrites history lines (#909)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation fixes and updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant